Integration Guide KUKA (LBR iiwa)
Note: This integration guide describes the Robot module for an older version of the communication interface (BPS 1.6). The module works with the current Bin Picking Studio (BPS) because of backward compatibility. It does not support features added later.
Note: We strongly recommend that you read the Robot communication overview before this integration guide.
Contents
1 Prerequisites
Prerequisites for the Robot module:
The Robot module was originally developed with Sunrise Cabinet firmware version V2.0.3-0 and Sunrise.OS software version . It should be compatible with all versions of Sunrise.OS.
To check your software version, go to Station -> Information.
2 Robot controller setup
2.1 Controller configuration
2.1.1 Network configuration
Make sure that the IP address of the Sunrise Cabinet matches your network configuration.
In your Sunrise Workbench project, open the StationSetup.cat file and go to the Configuration tab. Here you can change the IP address and subnet mask of your Sunrise Cabinet. After you edit the network configuration, save the file and synchronize the project with your controller.
Note: The Robot module uses TCP ports 54 601 and 30 004 for communication with the vision controller.
For more information, see the original KUKA documentation.
2.2 Robot module installation
The Robot module consists of the following files (folders are bold):
photoneo
Datatypes.java
Exceptions.java
PhotoneoCommon.java
StateServer.java
TrajectoryExecutor.java
photoneo_examples
Calibration.java
ChangeSolution.java
MultipleVisionSystems.java
SimpleBinpicking.java
2.2.1 Loading the Robot module files
Copy the whole photoneo folder (and, optionally, the example programs that you want to use) into the source directory of your project (<project>/src).
The project structure should look like the screenshot below.
Your project sources are now ready to be synchronized to the robot controller.
2.2.2 Robot State Server configuration
The Robot State Server must run so that you can calibrate a vision system, teach a gripping point, and use hand-eye vision systems. It also provides data for real-time visualization. The Robot State Server is implemented as a RoboticsAPICyclicBackgroundTask. When the file StateServer.java is loaded to the robot controller, the Robot State Server starts and runs automatically. No special setup is required.
After you load the StateServer.java source file to the robot controller, StateServer is shown as running (green control light) in the Background Tasks.
3 Robot module
The Robot module is designed for easy integration into existing applications written in Java.
3.1 Robotic API
Note: We strongly recommend that you read the Photoneo robotic API before this section.
3.1.1 Connection procedures
Warning: These procedures are contained in the PhotoneoCommon class and must not be edited!
Connection procedure |
Description / Usage |
|---|---|
Connect to Action Request Server boolean connectToVisionController(String address,int timeout) |
Description
Establishes a new connection to the Action Request Server.
Input parameters:
Return value:
Usage
Call the procedure only once, at the beginning of the program. You can send requests only after the connection is established.
|
3.1.2 Communication procedures
Note: For a detailed description of these procedures, see Action requests.
Warning: These procedures are contained in the PhotoneoCommon class (PhotoneoCommon.java) and must not be edited!
Bin picking requests
Request |
Input variables |
Output variables |
|---|---|---|
Initialization request SimpleResult initRequest(JointPosition end,JointPosition start,int visionSystemId) |
|
|
Scan request void scanRequest(int visionSystemId) |
|
Note: The Wait for scan completion procedure receives the response. |
Trajectory request void trajectoryRequest(int visionSystemId) |
|
Note: The Receive trajectory procedure receives the response. |
Pick-failed request SimpleResult sendPickFailed(int visionSystemId) |
|
|
Calibration requests
Request |
Input variables |
Output variables |
|---|---|---|
Add calibration point request SimpleResult calibrationAddPoint() |
— |
|
Solution requests
Request |
Input variables |
Output variables |
|---|---|---|
Change solution request SimpleResult changeSolutionRequest(int solutionId) |
|
|
Start solution request SimpleResult startSolutionRequest(int solutionId) |
|
|
Stop solution request SimpleResult stopSolutionRequest() |
— |
|
Get running solution request IntResult getRunningSolutionRequest() |
— |
|
Get available solutions request IntsResult getAvailableSolutionsRequest() |
— |
|
Response receiving procedures
Response receiving procedures |
Input variables |
Output variables |
|---|---|---|
Wait for scan completion SimpleResult waitForScan() |
— |
|
Receive trajectory TrajectoryResult receiveTrajectory() |
— |
|
3.1.3 Bin picking procedures
Note: These procedures are in the TrajectoryExecutor class (TrajectoryExecutor.java). Implement (edit) them as your application requires.
TrajectoryExecutor is a helper class that executes the received bin picking trajectory. It has two constructors:
TrajectoryExecutor(Robot robot)TrajectoryExecutor(Robot robot, double[] velocity, double[] acceleration)
Both constructors need an object of type Robot (you can also pass an object of type LBR). The optional velocity and acceleration arrays set the normalized velocities and accelerations of the motion (values from 0 to 1) for each trajectory segment of the bin picking routine.
You must override the gripper procedures to run your own code (set the gripper I/Os, and so on). The easiest way is to use an anonymous inner class, for example:
TrajectoryExecutor executor = new TrajectoryExecutor(robot) {
@Override
protected void gripperAttach() {
// insert your code here
}
};
Bin picking procedure |
Description / Usage |
|---|---|
Gripper attach void gripperAttach() |
Description
A user-defined procedure. Typically, it is the attach procedure for grasping the picked object in the Grasp waypoint.
Usage
It runs automatically when the robot reaches a grasping method waypoint that is configured to run the Attach procedure.
|
Gripper detach void gripperDetach() |
Description
A user-defined procedure. Typically, it is the detach procedure for releasing the picked object in the placing routine that the robot operator defines.
Usage
It runs automatically when the robot reaches a grasping method waypoint that is configured to run the Detach procedure.
Note: Typically, this procedure is not configured to run automatically in a waypoint. It is called during placing, which the robot operator implements. |
Gripper user-defined 1 void user1() |
Description
A user-defined procedure.
Usage
It runs automatically when the robot reaches a grasping method waypoint that is configured to run the User 1 procedure.
|
Gripper user-defined 2 void user2() |
Description
A user-defined procedure.
Usage
It runs automatically when the robot reaches a grasping method waypoint that is configured to run the User 2 procedure.
|
Gripper user-defined 3 void user3() |
Description
A user-defined procedure.
Usage
It runs automatically when the robot reaches a grasping method waypoint that is configured to run the User 3 procedure.
|
Execute bin picking routine IMotionContainer execute(TrajectoryResult trajectoryResult) |
Description
Predefined procedure that runs the bin picking routine. You must not edit this procedure directly. To adapt the execution settings, see Bin picking routine execution settings.
Input parameters:
Return value:
Usage
Run it after the bin picking trajectory is received. The robot must be in the start pose when the procedure starts. At the end of the procedure, the robot is in the end pose with the picked object attached to the gripper.
Note: The trajectory is executed asynchronously (Robot.moveAsync(…)) and an IMotionContainer object is returned. This lets you append other waypoints at its end, so the robot does not stop at the bin picking end position.
To wait until the whole bin picking trajectory is executed (before the next program instructions), call the method await() on the IMotionContainer object.
Warning: If you use multiple start poses (different for each vision system), make sure that the robot is in the correct one before you run this procedure. |
3.1.4 Request response types
Error error - error code
boolean hasError() - method that checks for an error (true if an error occurred)Error error - error code
boolean hasError() - method that checks for an error (true if an error occurred)
int toolPointInv - tool point invariance
int grippingPointId - gripping point ID
int grippingPointInv - gripping point invariance
Queue<RobotTask> robotTasks - computed bin picking trajectory, a queue of tasks for the TrajectoryExecutor [internal variable]
ArrayList<Integer> infoData - additional data [internal variable]Error error - error code
boolean hasError() - method that checks for an error (true if an error occurred)
Integer value - solution IDError error - error code
boolean hasError() - method that checks for an error (true if an error occurred)
Integer[] values - array of available solution IDs
3.2 Example programs
This section contains basic example programs. Each program is for a specific bin picking application and shows the correct use of the robotic API.
These templates also contain simple error handling. It is only an example. You must define your own routines for error situations.
3.2.1 Basic bin picking example
This program is a very basic example of a simple bin picking application. It connects to the vision controller and initializes one vision system. Then, in a loop, it requests a scan and a trajectory, and executes the received trajectory.
Name: SimpleBinpicking.java (in the photoneo_examples package)
package photoneo_examples;
import javax.inject.Inject;
import com.kuka.roboticsAPI.applicationModel.RoboticsAPIApplication;
import static com.kuka.roboticsAPI.motionModel.BasicMotions.*;
import com.kuka.roboticsAPI.deviceModel.JointPosition;
import com.kuka.roboticsAPI.deviceModel.LBR;
import com.kuka.roboticsAPI.uiModel.ApplicationDialogType;
import com.kuka.task.ITaskLogger;
import photoneo.PhotoneoCommon;
import photoneo.Datatypes.SimpleResult;
import photoneo.Datatypes.TrajectoryResult;
import photoneo.Datatypes;
import photoneo.TrajectoryExecutor;
public class SimpleBinpicking extends RoboticsAPIApplication {
@Inject
private LBR robot;
@Inject
protected ITaskLogger logger;
PhotoneoCommon phoCommon;
TrajectoryExecutor phoExecutor;
// ================= Update settings for your setup!!!
final String serverAddress = "";
final JointPosition placingPosition = new JointPosition(9, 9, 9, 9, 9, 9, 9);
final JointPosition start = new JointPosition(9, 9, 9, 9, 9, 9, 9); // Binpicking start position
final JointPosition end = new JointPosition(9, 9, 9, 9, 9, 9, 9); // Binpicking end position
final int vsId = 1; // vision system ID
// ================= Update settings for your setup!!!
public void initialize() {
// Instantiate required classes to do Binpicking
phoCommon = new PhotoneoCommon(robot);
phoExecutor = new TrajectoryExecutor(robot, new double[]{1, 0.5, 0.5, 1}, null) { // anonymous class
@Override
protected void gripperAttach() {
logger.info("Attaching gripper - Implement your I/O here");
}
};
}
public void run() {
// Connect to Binpicking Vision Controller
boolean success = phoCommon.connectToVisionController(serverAddress, 10000);
if (!success) {
logger.error("Unable to connect to Vision Controller. Check your network setup. Terminating program.");
return;
}
// Initialize Binpicking for desired vision system
SimpleResult simpleResult = phoCommon.initRequest(start, end, vsId);
if (simpleResult.hasError()) {
logger.error(simpleResult.error + " error occurred. It seems that Binpicking solution is not deployed. Terminating program.");
return;
}
// Prepare robot - move to placing position
robot.move(ptp(placingPosition));
// Scan and localize picked objects
phoCommon.scanRequest(vsId);
while (true) {
// Prepare robot to Binpicking start position
robot.move(ptp(start));
// Wait for finished scan and localization
SimpleResult scanResult = phoCommon.waitForScan();
if (scanResult.hasError()) {
if (doYouWantToContinue(scanResult.error)) { continue; } else { return;}
}
// Get planned trajectory
phoCommon.trajectoryRequest(vsId);
TrajectoryResult trajectoryResult = phoCommon.receiveTrajectory();
if (trajectoryResult.hasError()) {
if (doYouWantToContinue(trajectoryResult.error)) { continue; } else { return;}
}
// Execute Binpicking trajectory
phoExecutor.execute(trajectoryResult);
// Move from Binpicking end to placing position
robot.move(ptp(placingPosition));
// Perform new scan and localization during placing picked object
phoCommon.scanRequest(vsId);
logger.info("Detaching gripper"); // here you can set your I/O
}
}
/**
* This is helper function to do stuff around error handling
*
* @param error
* @return true if user want to continue in program flow and repeat program loop, false otherwise
*/
public boolean doYouWantToContinue(Datatypes.Error error) {
String errorText = error + " error occurred during program run. Do you want to continue?";
int terminateProgram = getApplicationUI().displayModalDialog(ApplicationDialogType.QUESTION, errorText, "Continue", "Terminate program");
if (terminateProgram == 1) { // Terminate option chosen
logger.info("Terminating program");
return false;
} else {
robot.move(ptp(placingPosition).setJointVelocityRel(0.2)); // set lower movement speed
phoCommon.scanRequest(vsId);
return true;
}
}
}
Program explanation
final JointPosition placingPosition = new JointPosition(9, 9, 9, 9, 9, 9, 9);
final JointPosition start = new JointPosition(9, 9, 9, 9, 9, 9, 9); // Binpicking start position
final JointPosition end = new JointPosition(9, 9, 9, 9, 9, 9, 9); // Binpicking end position
Declaration of the PhotoneoCommon and TrajectoryExecutor objects. They are defined in the initialize() method.
// ================= Update settings for your setup!!!
final String serverAddress = "";
final JointPosition placingPosition = new JointPosition(9, 9, 9, 9, 9, 9, 9);
final JointPosition start = new JointPosition(9, 9, 9, 9, 9, 9, 9); // Binpicking start position
final JointPosition end = new JointPosition(9, 9, 9, 9, 9, 9, 9); // Binpicking end position
final int vsId = 1; // vision system ID
// ================= Update settings for your setup!!!
Definitions of the variables used in the program. Adapt these settings to your application. Define joint positions in radians.
phoCommon = new PhotoneoCommon(robot);
Construction of a PhotoneoCommon object with the LBR robot object as a parameter.
phoExecutor = new TrajectoryExecutor(robot, new double[]{1, 0.5, 0.5, 1}, null) { // anonymous class
@Override
protected void gripperAttach() {
logger.info("Attaching gripper - Implement your I/O here");
}
};
Construction of a TrajectoryExecutor object that executes the received bin picking trajectory. This constructor also gets the LBR robot object, together with the speed parameters for the individual segments of the bin picking trajectory:
Start -> Approach (100%)
Approach -> Grasp (50%)
Grasp -> Deapproach (50%)
Deapproach -> End (100%)
The example uses the default machine accelerations, so null is passed as the third argument.
The anonymous class declaration lets you override the procedures that can run in a configured waypoint of the grasping method. Here, the Gripper attach method is overridden to log a string.
boolean success = phoCommon.connectToVisionController(serverAddress, 10000);
if (!success) {
logger.error("Unable to connect to Vision Controller. Check your network setup. Terminating program.");
return;
}
Next, the program tries to connect to the Action Request Server on the vision controller. If the connection is established, the variable success is true and the program continues. If the 10-second timeout is reached or another connection problem occurs, success is false and the program stops.
// Initialize Binpicking for desired vision system
SimpleResult simpleResult = phoCommon.initRequest(start, end, vsId);
Initialization of the vision system with the start/end bin picking poses (robot joint positions).
if (simpleResult.hasError()) {
logger.error(simpleResult.error + " error occurred. It seems that Binpicking solution is not deployed. Terminating program.");
return;
}
Check if an error occurred during the initialization request. If yes, the error is logged to the application output and the program stops.
// Prepare robot - move to placing position
robot.move(ptp(placingPosition));
Move the robot to the placing position that is out of the scanning area.
// Scan and localize picked objects
phoCommon.scanRequest(vsId);
Call the first scan request.
// Prepare robot to Binpicking start position
robot.move(ptp(start));
While the scan, localization, and path planning run, the robot moves to the bin picking start position.
// Wait for finished scan and localization
SimpleResult scanResult = phoCommon.waitForScan();
if (scanResult.hasError()) {
if (doYouWantToContinue(scanResult.error)) { continue; } else { return;}
}
When the robot is at the bin picking start position, the procedure Wait for scan completion receives the response to the scan request. If an error occurred during the scan, the error is handled.
// Get planned trajectory
phoCommon.trajectoryRequest(vsId);
TrajectoryResult trajectoryResult = phoCommon.receiveTrajectory();
if (trajectoryResult.hasError()) {
if (doYouWantToContinue(trajectoryResult.error)) { continue; } else { return;}
}
If the scan was successful, the program requests a bin picking trajectory with the Trajectory request and receives it with the procedure Receive trajectory. The error handling is the same as before.
// Execute Binpicking trajectory
phoExecutor.execute(trajectoryResult);
If the bin picking trajectory was received, the program executes it. Because the robot movement is asynchronous, the procedure Execute bin picking routine returns immediately. While the robot moves, the program continues to the next line:
// Move from Binpicking end to placing position
robot.move(ptp(placingPosition));
The robot finishes the bin picking routine and moves on to the placing position, where it stops.
// Perform new scan and localization during placing picked object
phoCommon.scanRequest(vsId);
When the robot is out of the scanning area, the program can call the next scan request.
logger.info("Detaching gripper"); // here you can set your I/O
While the scan runs, the robot can place the picked object. Add the gripper command here.
The program then continues from the beginning of the main loop.
3.2.2 Multiple Vision Systems example
This program extends the basic bin picking example. It initializes two vision systems instead of one and switches between them in each cycle.
Name: MultipleVisionSystems.java (in the photoneo_examples package)
package photoneo_examples;
import com.kuka.roboticsAPI.applicationModel.RoboticsAPIApplication;
import com.kuka.roboticsAPI.deviceModel.JointPosition;
import com.kuka.roboticsAPI.deviceModel.LBR;
import com.kuka.roboticsAPI.uiModel.ApplicationDialogType;
import com.kuka.task.ITaskLogger;
import photoneo.PhotoneoCommon;
import photoneo.Datatypes;
import photoneo.Datatypes.SimpleResult;
import photoneo.Datatypes.TrajectoryResult;
import photoneo.TrajectoryExecutor;
import javax.inject.Inject;
import static com.kuka.roboticsAPI.motionModel.BasicMotions.ptp;
/**
* Helper data structure to represent a vision system with required attributes
*/
class VisionSystemData {
int id;
JointPosition start;
JointPosition end;
VisionSystemData(int id, JointPosition start, JointPosition end) {
this.id = id;
this.start = start;
this.end = end;
}
}
public class MultipleVisionSystems extends RoboticsAPIApplication {
@Inject
private LBR robot;
@Inject
protected ITaskLogger logger;
PhotoneoCommon phoCommon;
TrajectoryExecutor phoExecutor;
// ================= Update settings for your setup!!!
final String serverAddress = "";
final JointPosition placingPosition = new JointPosition(9, 9, 9, 9, 9, 9, 9);
final VisionSystemData firstVs = new VisionSystemData(1,
new JointPosition(9, 9, 9, 9, 9, 9, 9),
new JointPosition(9, 9, 9, 9, 9, 9, 9));
final VisionSystemData secondVs = new VisionSystemData(2,
new JointPosition(9, 9, 9, 9, 9, 9, 9),
new JointPosition(9, 9, 9, 9, 9, 9, 9));
VisionSystemData activeVs = firstVs;
// ================= Update settings for your setup!!!
public void initialize() {
// Instantiate required classes to do Binpicking
phoCommon = new PhotoneoCommon(robot);
phoExecutor = new TrajectoryExecutor(robot) { // anonymous class
@Override
protected void gripperAttach() {
logger.info("Attaching gripper - Implement your I/O here");
}
};
}
public void run() {
// Connect to Binpicking Vision Controller
boolean success = phoCommon.connectToVisionController(serverAddress, 10000);
if (!success) {
logger.info("Unable to connect to Vision Controller. Check your network setup. Terminating program.");
return;
}
// Initialize Binpicking for desired vision system
phoCommon.initRequest(firstVs.start, firstVs.end, firstVs.id);
phoCommon.initRequest(secondVs.start, secondVs.end, secondVs.id);
// Prepare robot - move to placing position
robot.move(ptp(placingPosition));
// Scan and localize picked objects
phoCommon.scanRequest(activeVs.id);
while (true) {
// Prepare robot to Binpicking start position
robot.move(ptp(activeVs.start));
// Wait for finished scan and localization
SimpleResult scanResult = phoCommon.waitForScan();
if (scanResult.hasError()) {
if (doYouWantToContinue(scanResult.error)) { continue; } else { return;}
}
// Get planned trajectory
phoCommon.trajectoryRequest(activeVs.id);
TrajectoryResult trajectoryResult = phoCommon.receiveTrajectory();
if (trajectoryResult.hasError()) {
if (doYouWantToContinue(trajectoryResult.error)) { continue; } else { return;}
}
// Execute Binpicking trajectory
phoExecutor.execute(trajectoryResult);
// Move from Binpicking end to placing position
robot.move(ptp(placingPosition));
// Switch active vision system
activeVs = (activeVs.id == firstVs.id) ? secondVs : firstVs;
// Perform new scan and localization during placing picked object
phoCommon.scanRequest(activeVs.id);
logger.info("Detaching gripper"); // here you can set your I/O
}
}
/**
* This is helper function to do stuff around error handling
*
* @param error
* @return true if user want to continue in program flow and repeat program loop, false otherwise
*/
public boolean doYouWantToContinue(Datatypes.Error error) {
String errorText = error + " error occurred during program run. Do you want to continue?";
int terminateProgram = getApplicationUI().displayModalDialog(ApplicationDialogType.QUESTION, errorText, "Continue", "Terminate program");
if (terminateProgram == 1) { // Terminate option chosen
logger.info("Terminating program");
return false;
} else {
robot.move(ptp(placingPosition).setJointVelocityRel(0.2)); // set lower movement speed
phoCommon.scanRequest(activeVs.id);
return true;
}
}
}
3.2.3 Change solution example
One robotic cell can take part in several production processes. To handle multiple parts at the same time, use multiple vision systems in one solution. If you change the production process completely, it is better to have separate solutions. The robot can deploy them directly.
This program extends the basic bin picking example. After a defined number of bin picking cycles, it sends a request to change the deployed solution.
Name: ChangeSolution.java (in the photoneo_examples package)
package photoneo_examples;
import com.kuka.roboticsAPI.applicationModel.RoboticsAPIApplication;
import com.kuka.roboticsAPI.deviceModel.JointPosition;
import com.kuka.roboticsAPI.deviceModel.LBR;
import com.kuka.roboticsAPI.uiModel.ApplicationDialogType;
import com.kuka.task.ITaskLogger;
import photoneo.PhotoneoCommon;
import photoneo.Datatypes;
import photoneo.Datatypes.SimpleResult;
import photoneo.Datatypes.TrajectoryResult;
import photoneo.TrajectoryExecutor;
import javax.inject.Inject;
import static com.kuka.roboticsAPI.motionModel.BasicMotions.ptp;
public class ChangeSolution extends RoboticsAPIApplication {
@Inject
private LBR robot;
@Inject
protected ITaskLogger logger;
PhotoneoCommon phoCommon;
TrajectoryExecutor phoExecutor;
// ================= Update settings for your setup!!!
final String serverAddress = "";
final JointPosition placingPosition = new JointPosition(9, 9, 9, 9, 9, 9, 9);
final JointPosition start = new JointPosition(9, 9, 9, 9, 9, 9, 9); // Binpicking start position
final JointPosition end = new JointPosition(9, 9, 9, 9, 9, 9, 9); // Binpicking end position
final int vsId = 1; // vision system ID
final int firstSolutionId = 1;
final int secondSolutionId = 2;
int currentSolutionId = firstSolutionId; // start with the first solution
final int maxPicksPerSolution = 10;
int pickCounter = 0;
// ================= Update settings for your setup!!!
public void initialize() {
// Instantiate required classes to do Binpicking
phoCommon = new PhotoneoCommon(robot);
phoExecutor = new TrajectoryExecutor(robot) { // anonymous class
@Override
protected void gripperAttach() {
logger.info("Attaching gripper - Implement your I/O here");
}
};
}
public void run() {
// Connect to Binpicking Vision Controller, wait max 10 seconds for it
boolean success = phoCommon.connectToVisionController(serverAddress, 10000);
if (!success) {
logger.info("Unable to connect to Vision Controller. Check your network setup. Terminating program.");
return;
}
// Initialize Binpicking for desired vision system
phoCommon.initRequest(start, end, vsId);
// Prepare robot - move to placing position
robot.move(ptp(placingPosition));
// Scan and localize picked objects
phoCommon.scanRequest(vsId);
while (true) {
// Prepare robot to Binpicking start position
robot.move(ptp(start));
// Wait for finished scan and localization
SimpleResult scanResult = phoCommon.waitForScan();
if (scanResult.hasError()) {
if (doYouWantToContinue(scanResult.error)) { continue; } else { return;}
}
// Get planned trajectory
phoCommon.trajectoryRequest(vsId);
TrajectoryResult trajectoryResult = phoCommon.receiveTrajectory();
if (trajectoryResult.hasError()) {
if (doYouWantToContinue(trajectoryResult.error)) { continue; } else { return;}
}
// Execute Binpicking trajectory
phoExecutor.execute(trajectoryResult);
// Move from Binpicking end to placing position
robot.move(ptp(placingPosition));
pickCounter++;
if (pickCounter == maxPicksPerSolution) {
pickCounter = 0;
currentSolutionId = (currentSolutionId == firstSolutionId) ? secondSolutionId : firstSolutionId; // swap currentSolutionId
SimpleResult changeSolResult = phoCommon.changeSolutionRequest(currentSolutionId);
if (changeSolResult.hasError()) {
if (doYouWantToContinue(changeSolResult.error)) { continue; } else { return;}
}
// Initialize Binpicking for desired vision system
phoCommon.initRequest(start, end, vsId);
// Establish new connection to Binpicking Vision Controller, wait max 60 seconds for it
success = phoCommon.connectToVisionController(serverAddress, 60000);
if (!success) {
logger.info("Unable to connect to Vision Controller. Check your network setup. Terminating program.");
return;
}
}
// Perform new scan and localization during placing picked object
phoCommon.scanRequest(vsId);
logger.info("Detaching gripper"); // here you can set your I/O
}
}
/**
* This is helper function to do stuff around error handling
*
* @param error
* @return true if user want to continue in program flow and repeat program loop, false otherwise
*/
public boolean doYouWantToContinue(Datatypes.Error error) {
String errorText = error + " error occurred during program run. Do you want to continue?";
int terminateProgram = getApplicationUI().displayModalDialog(ApplicationDialogType.QUESTION, errorText, "Continue", "Terminate program");
if (terminateProgram == 1) { // Terminate option chosen
logger.info("Terminating program");
return false;
} else {
robot.move(ptp(placingPosition).setJointVelocityRel(0.2)); // set lower movement speed
phoCommon.scanRequest(vsId);
return true;
}
}
}
3.2.4 Calibration example
This program is a template for semi-automatic calibration.
Before you run the program:
Teach the calibration start pose. The robot moves through this pose to each calibration pose.
Teach the calibration poses.
The program contains 5 poses. Add more (at least 9 calibration poses are required).
Start the calibration in BPS.
Now you can start the program. The robot moves to each calibration pose and sends the Add calibration point request when it reaches the pose. Between the calibration poses, the robot moves through the calibration start pose. When all calibration points are added, the program ends. If you are satisfied with the calibration result, save it in BPS.
Name: Calibration.java (in the photoneo_examples package)
package photoneo_examples;
import com.kuka.roboticsAPI.applicationModel.RoboticsAPIApplication;
import com.kuka.roboticsAPI.deviceModel.JointPosition;
import com.kuka.roboticsAPI.deviceModel.LBR;
import com.kuka.roboticsAPI.geometricModel.ObjectFrame;
import com.kuka.roboticsAPI.uiModel.ApplicationDialogType;
import com.kuka.task.ITaskLogger;
import photoneo.PhotoneoCommon;
import photoneo.Datatypes.SimpleResult;
import javax.inject.Inject;
import static com.kuka.roboticsAPI.motionModel.BasicMotions.ptp;
public class Calibration extends RoboticsAPIApplication {
@Inject
private LBR robot;
@Inject
protected ITaskLogger logger;
PhotoneoCommon phoCommon;
// ================= Update settings for your setup!!!
final String serverAddress = "";
boolean popupAddQuestion = true;
// ================= Update settings for your setup!!!
// ================= Calibration poses are defined in `run():doc:` function
public void initialize() {
// Instantiate required classes to do Binpicking
phoCommon = new PhotoneoCommon(robot);
}
public void run() {
// Connect to Binpicking Vision Controller
boolean success = phoCommon.connectToVisionController(serverAddress, 10000);
if (!success) {
logger.info("Unable to connect to Vision Controller. Check your network setup. Terminating program.");
return;
}
// Define start pose for calibration!
// Position from which robot can safely get to all of calibration poses
ObjectFrame calibrationStart = getApplicationData().getFrame("/CalibrationStartPose");
// You can also define it via joint states
// JointPosition calibrationStart = new JointPosition(9, 9, 9, 9, 9, 9, 9);
// Define calibration poses!
ObjectFrame[] calibrationPoses = new ObjectFrame[]{
getApplicationData().getFrame("/Calibration1/P1"),
getApplicationData().getFrame("/Calibration1/P2"),
getApplicationData().getFrame("/Calibration1/P3"),
getApplicationData().getFrame("/Calibration1/P4"),
getApplicationData().getFrame("/Calibration1/P5")
};
// You can also define calibration poses via joint states
// JointPosition[] calibrationPoses = new JointPosition[] {
// new JointPosition(9, 9, 9, 9, 9, 9, 9),
// new JointPosition(9, 9, 9, 9, 9, 9, 9),
// };
for (int i = 0; i < calibrationPoses.length; i++) {
if (popupAddQuestion) {
int addQuestion = getApplicationUI().displayModalDialog(ApplicationDialogType.QUESTION,
"Do you want to add '" + i + "' calibration point?", "Add point", "Add all points", "Cancel");
if (addQuestion == 1) {
popupAddQuestion = false;
} else if (addQuestion == 2) {
return;
}
}
// Move to starting pose first - should be safe to move to any of calibration poses from this one
robot.move(ptp(calibrationStart));
// Move to i-th calibration pose
robot.move(ptp(calibrationPoses[i]));
SimpleResult result = phoCommon.calibrationAddPoint();
if (result.hasError()) {
getApplicationUI().displayModalDialog(ApplicationDialogType.ERROR,
"An error occurred while adding '" + i + "' point. Terminating program.");
return;
}
logger.info("Point '" + i + "' added successfully");
}
logger.info("All points added.");
}
}
3.3 Error handling & Exceptions
If an error occurs during the execution of the operation requested by the sent request the error is stored in the response object in the attribute error. We recommend that you add suitable error handling for your application after each synchronous request and response receiving procedure.
For the list of error codes with descriptions and troubleshooting, see Error codes.
The most important error codes are defined as constants in the enum Error in Datatypes.java. These error codes are:
Error code |
Java constant |
|---|---|
No error (0) |
NO_ERROR(0) |
Service error (1) |
SERVICE_ERROR(1) |
Path planning failed (201) |
PLANNING_FAILED(201) |
No object found (202) |
NO_PART_FOUND(202) |
Vision system not initialized (203) |
NOT_INITIALIZED(203) |
Empty scene (218) |
EMPTY_SCENE(218) |
Wrong bin picking configuration (255) |
WRONG_BPS_CONF(255) |
These exceptions can be thrown at runtime:
WrongUsage Thrown when the robotic API is used incorrectly. Review and fix the code.
ReceivedDataMismatch This exception matches the error Bad data (4) (it is implemented as an exception instead of an error enum).
CommunicationError This exception matches the error Communication error (3) (it is implemented as an exception instead of an error enum).
We do not recommend that you catch the WrongUsage and ReceivedDataMismatch exceptions in your program. If they occur, fix the problem. The CommunicationError exception can be caused by a power failure of the vision controller. In this case, you can catch the exception and try to connect to the Action Request Server again (after the vision controller starts).
4 Running the basic bin picking example program
4.1 Prerequisites
Before you can run the Basic bin picking example, make sure that:
A fully configured BPS solution with a single vision system is ready for deployment.
The robot controller is configured as described in Robot controller setup.
The BPS network settings are configured.
The gripper procedures are implemented (optional - without them, the robot does not actually pick the object).
The placing procedure is implemented.
4.2 Bin picking routine execution settings
As described in section 3.1.3 Bin picking procedures, you can pass optional velocity and acceleration arrays to the constructor of the TrajectoryExecutor class. They set the normalized velocities and accelerations of the motion (values from 0 to 1) for each trajectory segment of the bin picking routine.
By default, the grasping method of the BPS solution has 5 path stages with 4 trajectories between them. This means that you must pass arrays of 4 elements to the constructor of the TrajectoryExecutor class.
Adapt these values to your requirements. If you add custom path stages (trajectories), configure the matching number of values in the velocities / accelerations arrays. Keep the order of the trajectories: the first speed value in an array applies to the first trajectory, the second value to the second trajectory, and so on.
4.3 Reteach the robot poses
An important step of the bin picking configuration is to teach the home, start, and end poses. Teach the home position so that the robot is outside the scanning area. Teach the start position so that the robot gripper is approximately above the center of the bin. The end position can be similar to the start position or slightly shifted towards the placing area. Do not define the end pose too far from the bin. This can affect path planning (longer total planning time, planning errors, and so on).
The basic bin picking example program uses the following poses:
final JointPosition placingPosition = new JointPosition(9, 9, 9, 9, 9, 9, 9);
final JointPosition start = new JointPosition(9, 9, 9, 9, 9, 9, 9); // Binpicking start position
final JointPosition end = new JointPosition(9, 9, 9, 9, 9, 9, 9); // Binpicking end position
Note: In the basic bin picking example, the home pose is also the placing position (placingPosition).
4.4 Runtime
Deploy your BPS solution. On the Deployment page in BPS, the Action Request Client status should be DISCONNECTED (from the Action Request Server).
Start your robot program. After the procedure Connect to Action Request Server succeeds, the Action Request Client status on the Deployment page changes to CONNECTED.
The robot now starts to send requests to the vision controller and to execute bin picking movements.
Note: Be ready to stop the robot motion immediately if there is a problem. We strongly recommend that you reduce the speed to 10% of maximum or less during the first bin picking tests.